TITLE OF INVENTION 



System and method for voice commands recognition and controlling devices wirelessly using 
protocol based communication 



Inventor Name: Asim Hussain Abbasi 
Citizenship: Pakistan 

Residence: 110 Hidden Oaks Drive, Apt. 2C, 
Gary, North Carolina 27513, USA. 



1 



CROSS-REFERENCE TO RELATED APPLICATIONS 



Lots of wotk has been done in the field of voice recognition and numerous applications, 
systems have been developed so far based on Ihis technology. I like to put a reference to the 
following patents that contains the idea of voice commands and control, 

U.S. patent U.S. Pat* No. 6,1 88,985 entitled ^Wireless voice-activated device for control of a 
processor-based host system'' 

U.S. patent U.S. Pat. No. 5> 199,080 entitled "Voice-operated remote control system" 

U.S. patent U.S. Pat. No. 4,704,696 entitled "Method and apparatus for voice control of a 
computer" 

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT 
Not Applicable 

REFERENCE TO SEQUENCE LISTING, A TABLE OR A COMPUTER PROGRAM LISTING 
COMPACT DISK APPENDIX 
Not Applicable 

BACKGROUND OF INVENTION 

The present invention is related to using voice recognition engine, serial port interfacing, 
digital protocol writing, wireless communication, microcontroller pmgramming and design of 
switching circuits. 

Many different companies have developed voice recognition engines along with APIs 
(Application Programming Interfece) to allow programmers having no knowledge about DSP 



(Digital Signal ftrocessing) or voice recognition to embed high quality voice recognition ability 
into their softwares using APIs and develop applications based on it. 

Serial port interfacing describes how DTE (Data Terminal Equipment) communicates 
data serially using such ports on PC with DCE (Data Communication Equipment) and how to 
interface such ports with other electrical circuitry. Two serial ports are present in the form of DB- 
25 (called as Printer Port) and DB-9 in PC. 

Digital protocol writing requires the knowledge of electrical as well as software 
engineering. By writing protocol we actually develop a Language using which different systems 
(software/hardware) communicates with each other. 

Wireless communication describes the way to send bits into free-space throu^ 
transmitter and how to capture them correctly md convert them to bits again. These days many IC 
chip maker have developed single chip solutions for tiiis purpose Le. serial data transition over a 
range of 100 - +200meter at a rate of 10 - 20kbps and requires only PCB (Printed Circuit Board) 
febricated antenna. 

Microcontroller is a wonderfiil piece of device. U can define it as single chip computer 
and comprises of a processor capable of executing microcontroller specific Assembly Language, 
Parallel/Serial Ports, ROM (Read Only Memory), RAM (Random Access Memory), Registers 
(AX, BX, CX etc), SFR (Special Function Registers) etc. Microcontroller programmers use 
microcontroller specific simulators to write firmware. Simulators actually provides the software 
simulation of microcontroller's internal working thus helps the firmware writer to debug the 
firmware before actually sending the code into the microcontroller ROM using emulator. 

To control high voltage (120v) equipments especially inductive loads like Fan, water 
pump motor etc. requires special electrical circuitry to remove transients effects lhat result in 
switching of relays connected more specifically to such loads. Such circuits basically not only 
provides the interface between decision making device such as microcontroller and high voltage 
equipments but also helps to remove surges that induce in tfie overall electrical circuit due to 
switching relays* 



SUMMARY OF THE INVENTION 



Using this system, the entire home devices (Lamps, Fans etc) can be controlled through 
voice. The system comprises of a single PC, transmitter and receiver module attached with each 
device to be controlled. The user of this system will no need to move from his/her chair to turn 
on/off the light bulb or fen, moreover, when user has to go out from the home, there is no need for 
her/him to go to each room to turn off lights, fans etc. just by speaking voice commands will do all 
that. 

We will assign each receiver module two IDs, one will be Network^ID and other will be 
Device_ID. These IDs are assigned by system user using dip-switches on receiver module. In a 
particular network, every receiver module will have same Network_ID but different Device_IDs. 
After doing this, we will configure and synchronize the software having embedded voice 
recognition engine with these receiver modules* In the software we will assign each receiver 
module a unique name e.g. if we have attached one receiver module with fen as a device to be 
controlled then system user using the software can assign a name as "FAN" to it or ^jrfiatever. That 
name will then be used as voice commmd to control that device. Under running condition, vAen 
the ^stem user will speak *TAN" in the microphone, the software will recognize that voice 
command and will generate the corresponding Netwoik_ID and Device DD for that receiver 
module and will then build a packet of bits containing three frames as shown in Figure 4. That 
packet is then broadcasted through the transmitter attached with the serial port of the PC. Based on 
the protocol described in Figure 4A, all receiver modules will receive this packet but only the 
desired receiver module will respond to it. 



BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 shows the hardware setup of the voice controlled wireless system. There is a 
single transmitter attached with the serial port of PC and receiver module is attached with each 
device to be controlled. 
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Figure 2 describes how the network would be if we want to implement the same system to 
entire home. Audio transmitter and microphone unit has to be installed in each room^ There is a 
single audio receiver attached with microphone input on sound card of PC, Audio 
transmitter/receiver will work based on AM/FM technique whereas Digital transmitter/receiver 
will work based on ASK/FSK technique. 

Figure 3 depicts digital receiver module schematic with electric bulb attached as a device 
to be controlled. Two dip-switches are shown attached with the Microcontroller marked as 1 and 2. 
The receiver converts the signal (ASK/FSK) from transmitter back to bits. Those bits are fed to 
Microcontroller serially. Switching circuit receives the signal from Microcontroller and it contains 
a driving circuitry to control the relay. 

Figure 4A shows the protocol packet architecture used in the wireless communication 
between single trananitter with all receiver modules. Network_ID is transmitted first Ihen 
Device_E) and on last Command frame. 

Figure 4B describes the real world example of protocol packet where 4 bits are used for 
Network_ID frame, 8 bits are used for Device_E) frame and 4 bits are used for Command frame. 
In each frame, LSB (Least Si^ificant Bit) is transmitted first. 

Figure 5 tells the working of firmwatie inside the Microcontroller to implement protocol- 
Figure 5C shows the data read from 4-bit dip-switch meant for Networic_ID, is stored in BX 
register, with BL containing 001 L 

Figure 5D shows the data read from 8-bit dip-switch meant for Device D) is stored in CX 
register of Microcontroller having value 1 1 10001 L 

Figure 5 A depicts that data is coming into the Microcontroller in AX register serially 
using scheme "Serial-in, Serial-ouf Microcontroller will first look for Network JD and perform a 
comparison between AL and BL after in-takmg each bit. 



Figure 5B shows that after having a Network_ID match, the Microcontroller will intake 4 
bits and then perform the comparison between AX and CX 

Figure 5E shows that after having DeviceJD match, the Microcontroller will intake 4 bits 
and check AH. If its contents are 0001 then it means, command to tum-on the attached device (e.g. 
Electric Bulb) and if its contents are 0000 then it means, command to turn-off the attached device. 

DETAILED DESCRIPTION OF THE INVENTION 

This system consists mainly of four parts, software (runs on PC), transmitter, receiver 
(containing firmware in Microcontroller) and protocol. All of them are discussed one by one in the 
following paragraphs. 

Software will run on the PC and having voice commands recognition abilify and 
assigning each device to be controlled a unique name (e.g. Fan, Lamp etc. or vsiiatever as desired 
by the system user). When the system user wants to turn on the fen, he just has to say *Tan" in the 
microphone attached with the PC as shown in Figure 1 , Protocol has been developed and 
implemented in this system that describes how single transmitter vdll communicate with the 
receiver module attached v^th each device to be controlled. Based on this protocol^ software will 
send three frames of bits namely, Network_ID, Device_ID and Command as shown in Figure 4A 
to the transmitter attached with the serial port of the PC. 

Transmitter attached with the serial port of the PC will transmit those pulses based on the 
ASK (Amplitude Shift Keying) technique. Baud rate of serial port must be matched or at least less 
than max baud rate of the transmitter. There are chips from different brands that are single chip 
transmitter meant to be used with the serial port and provides a range of 1 00+ meters with baud 
rate of 1 0 or 20kbits/sec or even more. As in our case, we just want to turn devices on or off so 
having a system with baud rate of lOkbits/sec is more than enough. 

Receiver module attached with each device (Fan, Lamp etc.) will receive those signals 
and convert them back to bits. There must be two dip-switches (4/8 bits or wdmtever depends upon 
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the number of devices to be controlled) on flie receiver. One is meant for setting the Network_ID 
of the receiver and other for Device_ID. 

Protocol comprises of three frames of bits as shown in Figure 4A and describes the way 
how software after recognizing the voice command will broadcast the command through the 
trananitter and how only the desired receiver will respond to that command. 

Lets me first start with receiver module. The receiver module is ^own in the Figure 3 
with electric bulb attached. As is depicted in the Figure 3, the receiver module will contain two 
dip-switches. Using one we can assign Device_ID to that receiver so that it can recognize that this 
command sent by transmitter is for it to execute. Other will be used to assign the Network_ID e.g. 
if u want to control 12 devices using same toananitter then all receiver modules attached with each 
device would have same Network_ID (same dip-switch setting) but each would be having its own 
unique Device_ID (different dq)-switch setting). If u want to design the system to control 256 
devices then u will require 8-bits dip-switch for aUotting each device a unique Device_ID. 
Similar^ if u have two or more such networks running in the vfcinity of each other then for each 
receiver module to identify its own network trmsmitter, we will need dip-switch of more than 2 
bits on the each receiver module. 

Either FSK (Frequency Shift Keying) or ASK (Amplitude Shift Keying) or any other 
method of sending digital date can be used through transmitter. The receiver will receive those 
waves converts them back to bits and those bits will be read by Microcontroller through serial port. 

Transmitter will first send the Network_ID with least significant bit first. Receiver 
modules wiU first check whether it is a call fi-om the teansmitter on their network or some other 
network by comparing the NetwoikJD set on the receiver using dip-switch with the one it 
received from the transmitter. If both are same then it will look for the Device_ID and compares 
the one it received from the trananitter witfi tiie one system user dlotted using dip-switch on ti»e 
receiver. If Device_ID comparison also resulte in a match then the Microcontroller will look for 
the Command frame. If it is 0001 then it will turn on the attached device and if it is 0000 then it 
wiUtumito£ 



Now consider a network as shown in Figure 1 witfi one receiver module attached with the 
fan and other one with the electric bulb. Lets suppose we have set the dip-switch meant for 
Network_ID as 001 1 on both the receiver modules since both are to be controlled by same 
transmitter and Device_ID for electric bulb as 1 1 lOOOl 1 and for fen as 1 100001 1 using dip-switch 
meant for setting Device JD, Both of these dip-switches are depicted in Figure 3. After running 
the softwwe on the PC, s>^m user will first configure and synchronize the software with flie 
transmitter and receiver modules by first entering the Network_ID as set on each receiver module. 
Then system user will enter the Device JOD of each receiver module and assign each device a 
name, which will become tiie voice command for ttiat device e.g. if the receiver having DeviceED 
of 1 1 10001 1 is attached with a electric bulb then system user will write BULB (or whatever name 
he wants to assign to that device) in the name text box next to Device lD of the 1 1 10001 1 . In the 
same maimer system user will enter the Device_ID of each receiver module and assign a unique 
name to each device. 

When the system user will activate the software and will speak BULB in the microphone, 
the voice recognition engine embedded in the software will recogjiize the voice command and 
software will then come to know which Device JD system user wants to toggle (turn On/OflE)- The 
software will then build three frames of bits as shown in Figure 4B and will send it to transmitter 
for broadcast All receiver modules attached with each device vdll receiver those bits but only the 
desired receiver module will execute the command. 

In the receiver module, the received signal from transmitter is converted back to bits and 
those bits are fed to Microcontroller serially. The working of firmware in Microcontroller is 
explained in Figure 5. As shown in Figure 5C, the dip-switch data meant for Network lD is saved 
in BX register with BL containing 001 1 and the dip-switch data meant for Device_ID is saved in 
CX register as shown in Figure 5D having value 1 1 10001 1 . Hie received data is fed serially to AX 
register of MiciDController with scheme "Serial-in, Serial-out" e.g. as shown in Figure 5A, 1 will 
take the position of MSB (Most Significant Bit) and reset of the data in the register is shifted to the 
right one bit with LSB (Least Significant Bit: in this case 0) is taken out of Ihe register. 

The Microcontroller will first look for Network_ID and will run the comparison routine 
after each bit in-take into the AX register. It will keep comparing the value in the AL regster with 



8 



the value in the BL re^ster as shown in Figure 5A after each bit intake, unless a match is found. If 
the match is found, then the Microcontroller will look for Device_ID and will intake 4 bits tiie 
same way but without comparing the data in AX with data in CX after each bit intake. Because 
when AL results in a match with BL, the value in AH actualfy contains the 4 bits of Device_ID 
frame. After 4 bits intake, Ae Microcontrolter will Aen perform the comparison between AX and 
CX as shown in Figure 5B. If the match is found, then the Microcontroller will look for Command 
frame otherwise it will go back to main routine of looking for Network ID, as this is not the 
command for it. 



If AX and CX results in a match as in Figure 5B, then the Microcontroller will look for 
Command frame mid will intake 4 bits and will check the content of AH as shown in Figure 5C. 
If it is 0001 then its a command to turn on the attached device and if it is 0000 then its a 
command to turn oflF the attached device. Based on the information in the AH register. 
Microcontroller will then send the order to the switching circuit meant for driving reky as shown 



in Figure 3. 



If we have to extend the same system for complete home then we have to use one audio 
'^1 transmitter/receiver (using AM/FM technique) as well, as shown in Figure 2. One microphone 

and audio transmitter module is installed in each room. When the system user will speak 
y '"Bedroom Fan" the audio signal is transmitted and will receive by audio receiver attached with 

f I the microphone input of PC as shown in Figure 2. The voice engine embedded in the software 

2 will then recognize this voice command and software will build the frames based on the 

j'l Device_ID related to voice command, "Bedroom Fan". Those flames of bits are broadcasted 

through the transmitter. Based on that protocol as shown in Figure 4A only the desired device 
will respond to the voice command. 
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